Towards Transparent Systems: Semantic Characterization of Failure Modes

In this document, we give a more detailed explanation of the accuracy (ACC) vs. frequency-of-use (FOU) metric introduced in the main paper. We show the ACC vs. FOU curves for a human user1 as well as automatic failure prediction using our specification sheets. The area under these curves were reported in Tables 2 and 3 in the main paper (and are replicated here in Tables 1 and 2). Recall that one use case for our proposed specification sheets is the following: a user of a system can use the specification sheet to decide when to trust the system, and when not to trust it. If an input image falls in one of the scenarios listed in the specification sheet, the user would not trust the system. In other words, the user would not use the system for that input image, because according to the specification sheet, the system is like to fail on this input image anyway. Clearly, a specification sheet that lists many scenarios on it will capture many of the failure modes and perhaps (incorrectly capture) several scenarios that are not failures. As a result, the user will not be able to use the system very often (i.e. low frequency-of-use) which may be annoying or counter-productive for the user. But whenever he does use the system, it will likely be very accurate. This trade-off between the accuracy of the system (ACC) and frequency-of-use (FOU) is thus very relevant to the user – perhaps more so than the traditional precision-recall trade-off (results for the latter are shown in the paper, and may be more relevant for the use case where researchers are using our specification sheets to better understand their systems). If the input pre-trained classification system whose mistakes we are characterizing has a classification accuracy of 70%, a perfect specification sheet would ensure that all the mistakes (30% of the images) are detected by it, while the remaining 70% of the images are left un-flagged. Hence, a user using the specification sheet would ignore the vision system’s response (i.e. not use it) 30% of the time. Therefore, anywhere from 0 to 0.7 FOU, the vision system would have 100% accuracy. If the user chooses to work with a specification sheet that flags only 15% of the images (i.e. user uses the system 85% of the time), the accuracy of the vision system would be lower, somewhere between 70% and 100% at 0.85 FOU. If the user insists on ignoring the specification sheets and uses the system 100% of the time, the accuracy of the system will naturally be 70% (at FOU = 1.0). To capture this trade-off between accuracy of the system when used (ACC) and frequency-of-use (FOU), we plot ACC vs. FOU curves as shown in Figures 1 and 2.